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Abstract 



\ In this note, we prove a quantization formula for singular reduc- 

tions. The main result is obtained as a simple application of an ex- 
q . tended quantization formula proved in [TZ2]. 

O ■ 50. Introduction 

ON . 

""^ . Let (M, cu) be a closed symplectic manifold. We make the assumption that 

there is a Hermitian line bundle L over M admitting a Hermitian connection 
(3JT); V L with the property that ^^(V L ) 2 = cj. When such a line bundle exists, 

it is usually called a prequantum line bundle over M. Let J be an almost 
complex structure on TM so that g™(u, v) = u(u, Jv) defines a Riemannian 
metric on TM. Then one can construct canonically a Spin c -Dirac operator 

D L : fi°'*(M, L) -> fi°'*(M, L), (0.1) 

which gives the finite dimensional virtual vector space 

Q(M, L) = (kerD L ) n fi°' cvcn (M,L) - (ker D L ) n fi°' odd (M,L). (0.2) 

Next, suppose that (M, w) admits a Hamiltonian action of a compact con- 
nected Lie group G with Lie algebra g. Let /x : M — > g* be the corresponding 
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moment map. Then a formula due to Kostant [Ko] (cf. [TZ2, (1.13)]) in- 
duces a natural g action on L. We make the assumption that this g action 
can be lifted to a G action on L. Then this G action preserves V L . One can 
also assume, after an integration over G if necessary, that G preserves the 
Hermitian metric on L, the almost complex structure J and thus also the 
Riemannian metric g™. Q(M, L) then becomes a virtual representation of 
G. We denote by Q(M, L) G its G-invariant part. 

Now let a G g* be a regular value of \i. Let O a C g* be the coadjoint 
orbit of a. For simplicity, we assume that G acts on /i _1 ((9 a ) freely. Then 
the quotient space M G , a = n^iOa) /G is smooth. Furthermore, u descends 
canonically to a symplectic form uo a on M G , a so that one gets the Marsden- 
Weinstein reduction (MQ ta ,(jj a ). The pair (L, V L ) also descends canonically 
to a Hermitian line bundle La, a with a Hermitian connection denoted by 
V Lc ' a . [] The almost complex structure J also descends canonically to an 
almost complex structure on TMa, a - Thus once again one can construct 
canonically a Spin c -Dirac operator D L °' a as well as the corresponding virtual 
vector space Q(M G>a , L G , a ). 

The purpose of this short note is to give a direct proof of the following 
result as an immediate consequence of a result proved in [TZ2]. 

Theorem 0.1. 7//i _1 (0) is nonempty, then there exists an open neigh- 
borhood O of G g* such that for any regular value a G O of fi with /i _1 (a) 
nonempty, the following identity holds, 

dim Q(M, L) G = dimQ(M G , a , L G>a ). (0.3) 

If G g* is a regular value of //, then one can take a = in (0.3). This is 
the Guillemin- Sternberg geometric quantization conjecture [GS], which was 
proved in various generalities in [DGMW, GS, G, JK, Ml, M2, VI, V2, TZ1, 
TZ2]. Thus the main feature of Theorem 0.1 is that it equally applies when 
G g* is a singular value of \i. 

A complete treatment of Theorem 0.1 when G is the circle group was 
given in [TZ3] using the spectral flow technique. A different treatment of 

however, if a ^ 0, then Lq m is in general no longer a prequantum line bundle over 

(M G ,a,UG,a)- 
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(0.3) which works for nonabelian actions has also been given by Meinrenken 
and Sjamaar [MS]. 

Theorem 0.1 still holds, for orbifolds, when G does not act freely on 
// _1 ((!) a ). Furthermore, if (M, u) is Kahler and G acts on M holomorphically, 
then we can refine (0.3) to a system of holomorphic Morse type inequalities 
as in [TZ1, 2]. 

This note is organized as follows. In Section 1, we recall the result in 
[TZ2] to be used in this paper. In Section 2, we prove a key estimate which 
allows one to obtain Theorem 0.1 immediately. 



§1. An extended quantization formula 

We make the same assumption and use the same notation as in Introduc- 
tion. We further assume in this section that G g* is a regular value of /i 
and, for simplicity, that G acts on /x _1 (0) freely. Then one can construct the 
canonical Marsden-Weinstein reduction {Mq = fi~ 1 (0)/G, ooq)- 

We equip g (and thus g* also) with an Ad G-invariant metric. Let /ij, 
1 < % < dimG, be an orthonormal base of g*. Let V i: 1 < % < dimG, be the 
dual base of hi, 1 < i < dimG. Then the moment map \x can be written as 

dim G 

fj, = Mi, ( L1 ) 

i=l 

with each \ii a real function on M. 
Let 

H = \fi\ 2 (1.2) 

be the norm square of the moment map. It is clearly G-invariant. 

For any V E g, we use the same notation to denote the vector field it 
generates on M. 

Let now E be a Hermitian vector bundle over M admitting a Hermitian 
connection V s . We make the assumption that the G action on M lifts to 
a G action on E which preserves the Hermitian metric and the Hermitian 
connection V E on E. Then E descends canonically to a Hermitian vector 
bundle Eq over Mq with Hermitian connection V E °. 

With these data one can construct canonically a Spin c -Dirac operator 
D E acting on £f>*(M,E) = T(A '*(T*M) ® E) as well as the corresponding 



3 



virtual vector space Q(M,E) as in (0.2), with L there replaced now by E 
(cf. [TZ2, Sect. 1]). Similarly, one can construct the Spin c -Dirac operator 
D Eg on M G as well as the corresponding virtual vector space Q(M G , E G ). 
Furthermore, one verifies directly that the G action commutes with D E so 
that Q(M,E) is a virtual G-representation. Denote its G-invariant part by 
Q(M,E) G . 

Let V Eg. Set 

r* = L*-V*, (1.3) 

where Ly denotes the infinitesimal action of V on E. 

The following result was proved in Tian-Zhang [TZ2], where a direct ana- 
lytic approach to the Guillemin-Sternberg geometric quantization conjecture 
[GS] was developed. 

Theorem 1.1. ([TZ2, Theorem 4.2]). // /i _1 (0) is nonempty and if at 
each critical point x G M\/x _1 (0) ofTC, 

dimG 

v^T ]T to(x)r%(x)>0, (1.4) 
i=i 

t/ien the following identity holds, 

dimQ(M, Ef = dimQ(M G , S G ). (1.5) 



§2. A proof of Theorem 0.1 

In this section, we prove Theorem 0.1 by using Theorem 1.1. In order to 
explain the basic idea clearer, we will first treat the case where G is abelian 
in detail. Then as will be seen, the basic estimate in the G nonabelian case, 
which is necessary for applying Theorem 1.1, can be reduced to the abelian 
case. 

This section is organized as follows. In a), we deal with the G abelian 
case. In b), we prove Theorem 0.1 for the G nonabelian case. The final 
subsection c) contains some further remarks related to Theorem 0.1. 

a) The abelian case 
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In this subsection, we assume that G is a torus. 
For any a E g*, set 

H a = H-a. (2.1) 
Then /i a is again a moment map for the G- action on (M,cu). Let 

H a = K| 2 (2.2) 

be the norm square of /i a . 

Recall that h i: 1 < % < dimG, is an ortho normal base of g*. Thus a has 
the expression 

dim G 
i=l 

The main result of this subsection can be stated as follows. 

Proposition 2.1. There is an open neighborhood O o/OGg* such that 
for any a E O, the following inequality holds at each critical point ofH a , 

dimG 

E - a i)^ > °- ( 2 - 4 ) 

i=i 

Proof. We will use Kirwan's geometric characterization [Kl] of the critical 
point set of the norm square of moment maps to prove (2.4). 

As in [Kl, 3.4], denote by A C g* the finite set of weights associated to 
fi, which consists of the values of /i taking on the fixed point set of G on M. 
We also denote by Y the set of convex hulls in g* generated by nonempty 
subsets of A. Then Y consists naturally of two parts: the part Yj consists 
of all those convex hulls not containing 0, and the rest part denoted by Yjj. 

Now let Us be an open ball in g* with center and radius 5 > such that 
the closure Us does not intersect with any convex hull in Yj. The existence 
of Us is clear. Set O = Us 12 

Let a G O. Let A a = {A — a : A E A} be the finite set of weights 
associated to fi a , and Y a = {Y — a : Y E Y} the associated set of convex 
hulls. Then Y a consists of two parts accordingly: Yj a = {Y — a : Y E Y/} 
and Y IIa = {Y — a : Y E Y n }. One verifies easily that the closure O does 
not intersect with any convex hull in Y/ ifl . 
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Let B a be the open ball B a = {y G g* : \y + § | < |f |}. Clearly, B a C O. 
Thus 5 a does not intersect with any convex hull in Y^ a . 

Now take F G Y// j0 . Let y G F be the (unique) point on F which is 
closest to 0. We claim that y G g*\B a . 

To prove this claim, we suppose on the contrary that y £ B a . Let x G 9S a , 
with x 7^ —a, lie in the straight line generated by y and —a G dB a H F, then 
it is easy to see that rr G F. For if x ^ F, then y should lie in a face of F 
which does not contain —a. This would imply that y lies in a convex hull in 
Y/ i(1 , a contradiction. But with such an x G <9.B a nF, one encounters another 
contradiction d(0,x) < d(0,y). Thus we should have y ^ B a . 

By this and by the result of Kirwan [Kl, 3.12], one finds that if y is a 
critical point of TC a , then 

a a dimG 

o < \fi a (y) + - Id 2 = E (^(y) - (2-5) 

A z 1=1 

which is exactly (2.4). □ 

We can now prove Theorem 0.1 in this abelian case. In fact, the Kostant 
formula [Ko] (cf. [TZ2, (1.13)]) implies, in using the notation of (1.3), that 
for each 1 < % < dim G, 

r£. = -2nV-L^- (2.6) 
Thus one verifies via Proposition 2.1 that at any critical point of Tt a , one has 

dim G dim G 

v 7 ^! E (l* - a *) r k = 2vr E - > 0. (2.7) 

i=i i=i 

Theorem 0.1 then follows by applying Theorem 1.1 to (M, u, /i a , L). □ 



b) The nonabelian case 

In this subsection, we no longer assume that G is abelian. 

Let T be a maximal torus of G, with Lie algebra t. Then /j,t = PtH, 
where P T is the orthogonal projection from g* to t*, is the moment map of 
the induced T action on (M,lu) (cf. [Kl, 3.3]). Let O t C t* be the open set 
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defined in a) for lit- Then O = AdG(Or) C g* is an open neighborhood of 

oeg*. 

Let a G O be a regular value of 11 and, for simplicity, assume that G acts 
on ji^iOg) freely. Recall that the coadjoint orbit O a admits a canonical 
symplectic (actually Kahler) form uj a and the Ad G action on O a is Hamilto- 
nian with the moment map given by the canonical embedding i a : O a <^-> g* 
(cf. [McS, Chap. 5]). 

We now form the symplectic product (M x O a ,u x (—uj a )). Then the 
induced action of G on M x O a is Hamiltonian with the moment map fx : 
MxO^g* given by 

fx(x,b) = fx(x) -b. (2.8) 

Then G g* is a regular value of fx and G acts on /i _1 (0) freely. Furthermore, 
one has the standard identification of the symplectic quotients 

fi-\0)/G = ti-\O a )/G = M Gia (2.9) 

(cf. [McS, Chap. 5]). 
Let 

n = \fi\ 2 (2.io) 

be the norm square of the moment map fx. 

We now state the nonabelian extension of Proposition 2.1 as follows. 

Proposition 2.2. If (x,b) G M x O a is a critical point ofH, then one 
has the following inequality for the inner product on g*, 

(fx(x)-b,fx(x))>0. (2.11) 



Proof. Without loss of generality, we can assume that a G Or- By a result 
of Kirwan [K2, pp. 551], we know that for any critical point (x, b) G M x O a 
of H , one can find y G t* such that (y, a) is a critical point of H in the 
G-orbit of (x, b). 

Now since a G Ot, one finds from Proposition 2.1 and from Kirwan [Kl, 
3.3] that 

(tx(y)-a,Li(y))>0. (2.12) 
(2.11) then follows from the Ad G-invariance of the inner product on g*. □ 
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Proof of Theorem 0.1. Denote by tt the projection from M x O a to its 
first factor M. Let C = it*L be the pull-back Hermitian line bundle over 
M x O a with the pull-back Hermitian connection V L on C verifying that 

[ (V £ ) 2 (2.13) 



2tt 

Furthermore, the G action on L lifts canonically to an action on C. In 
particular, for any V G g, its induced infinitesimal action on C is, via the 
Kostant formula [Ko] (cf. [TZ2, (1.13)]) for the g-action on L, given by 

L£ = V$-2nV=l((m,V), (2.14) 

from which one has, in using the notation in (1.3), that 

r£. = ^Trv^ifc) (2.15) 

at any point (x, 6) G Mx0 a . Thus by (2.15) and Proposition 2.2, one verifies 
that at any critical point (x, b) G M x O a of 7Y, 

dim G 

V^lil t (x,b)ri(x,b) = 27,( f i(x)-b, f i(x))>0. (2.16) 

i=i 

One the other hand, one verifies directly that the induced line bundle Lq 
over /t _1 (0)/G is exactly the line bundle L G>a over M Gj(l = ^~ l (O a )/G. 
One can then apply Theorem 1.1 to M x O a , ft and £ to get 

dimQ(M x O a ,Cf = dim Q(M G ^L G>a ). (2.17) 

Furthermore, by the definition of £, one verifies directly that 

dimQ(Mx C a ,£) G = dimg(M,L) G -dimg(C a ,C) G , (2.18) 

with 

dimQ(a,C) G = l, (2.19) 

which can also be verified directly. (0.3) follows from (2.17)-(2.19). □ 

c) Further extensions and remarks 
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First of all, as have already been remarked in Introduction, Theorem 0.1 
still holds for orbifolds when G does not act on ^ l (O a ) freely, and that 
in the holomorphic category one can refine (0.3) to a system of Morse type 
inequalities. We will not fill the easy details here. 

The second point we want to remark is that by now it is clear that The- 
orem 1.1 has a very wide range of applicability, and Theorem 0.1 is just one 
manifestation of this. One can easily state and prove an extended version of 
Theorem 0.1 which works for general coefficients. In particular, one can take 
the trivial line bundle as coefficient to get as in [TZ2] that the Todd genus 
of M equals to the Todd genus of Mq^ for any regular value a of /i. This 
last result is also contained in [MS]. 

Last but not least, recall that we have extended the Guillemin-Sternberg 
geometric quantization conjecture [GS] to the case of manifolds with bound- 
ary in [TZ3]. Combining the methods there with our proof of Theorem 0.1, 
one can easily extend Theorem 0.1 to the case of manifolds with boundary 
as well. We again leave the details to the interested reader. 
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